System and method for holographic image-guided non-vascular perctaneous procedures

ABSTRACT

Holographic image-guidance can be used to track an interventional device during a non-vascular percutaneous procedure. The holographic image guidance can be provided by a head-mounted device by transforming tracking data and body image data to a common coordinate system and creating a holographic display relative to a patient&#39;s body to track the interventional device during the non-vascular percutaneous procedure. The holographic display can also include graphics to provide guidance for the physical interventional device as it travels through the patient&#39;s anatomy.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 15/958,711, entitled“SYSTEM AND METHOD FOR HOLOGRAPHIC IMAGE-GUIDED NON-VASCULARPERCUTANEOUS PROCEDURES”, filed Apr. 20, 2018, which claims the benefitof U.S. Provisional Application No. 62/487,519, filed April 20, 2017,entitled “3D HOLOGRAPHIC GUIDANCE AND DEVICE NAVIGATION AUGMENTED TO THEINTERVENTIONAL SITE FOR PERCUTANEOUS PROCEDURES”. These applications arehereby incorporated by reference in their entirety for all purposes.

GOVERNMENT FUNDING

This invention was made with government support under HL119810 awardedby the National Institutes of Health. The government has certain rightsin the invention.

TECHNICAL FIELD

The present disclosure relates generally to non-vascular percutaneousprocedures and, more specifically, to systems and methods that provideholographic image-guidance for the non-vascular percutaneous procedures.

BACKGROUND

Image-guided surgery (IGS), also known as surgical navigation, visuallycorrelates intraoperative anatomy with a preoperative image in real-time(or “live”). Often, IGS is considered to be analogous to global positionsystem (GPS), a technology that permits individuals to show theirrelative position on a computer-generated map. In IGS, the preoperativeimage can serve as the map, and the intraoperative tracking system issimilar to the satellites and devices that are used for GPS. Using IGSprovides greater control of a surgical procedure, real-time feedback onthe effect of the intervention, and reduced trauma/disruption whenaccessing the surgical target. Accordingly, IGS is often used for biopsyand other non-vascular interventional procedures using a needle or otherinterventional instrument.

The theoretical usefulness of IGS is limited in practice due to thevisual correlation of the intraoperative anatomy with the preoperativeimage. Increased use of the intraoperative imaging would lead to greaterconfidence with avoiding critical structures and locating the target,but this leads to an increased radiation dose burden to the patient andthe interventionist due to the real time fluoroscopy or computedtomography (CT). Additionally, images of the target and the needle orother interventional instrument are presently displayed on a flat, 2Dmonitor at tableside. To control the needle or other interventionalinstrument, the interventionist must translate its position andtrajectory relative to the target viewed on a 2D monitor into physicaltrajectory adjustments that are needed to correct the path of theinstrument. Current image-guidance techniques can lead to procedurerelated complications (such as pneumothorax, in the case of lung nodulebiopsy, or hemorrhage associated with liver biopsy). Moreover, the useof CT guidance for percutaneous procedures can affect revenue for theinstitution by reducing the number of diagnostic scans being performed(decreasing throughput). A typical CT guided biopsy procedure requiresabout one hour in the CT scanner suite, which distracts from its use inperforming diagnostic imaging procedures. On average, four diagnostic CTprocedures can be performed in the time it takes to complete one biopsy(˜1 hour), translating directly to lost revenue.

SUMMARY

The present disclosure relates generally to non-vascular percutaneousprocedures and, more specifically, to systems and methods that provideholographic image-guidance for the non-vascular percutaneous procedures.

In one aspect, the present disclosure can include a method for providingholographic image-guidance for a non-vascular percutaneous procedure.The method can be performed by a head-mounted device that includes aprocessor, which can receive tracking data for a physical interventionaldevice in a tracking coordinate system; transform the tracking data forthe physical interventional device in the tracking coordinate systeminto a headset coordinate system; access image data from a pre-operativeimage of a patient's anatomy comprising a physical operative site in animaging coordinate system; transform the image data in the imagingcoordinate system into the headset coordinate system; register a 3Dholographic representation of the interventional device based on thetracking data for the physical interventional device in the headsetcoordinate system to 3D anatomical holographic projections of thepatient's anatomy based on the imaging data in the headset coordinatesystem; display the 3D anatomical holographic projections providing avisualization of a holographic version of the patient's anatomy withreference graphics related to a physical operative site within thepatient's anatomy; display the 3D holographic representation of theinterventional device providing a visualization of a holographic versionof the interventional device with guidance control graphics related tothe physical interventional device; and navigate the 3D holographicrepresentation of the interventional device in the 3D anatomicalholographic projections based on the tracking data for theinterventional device in the headset coordinate system. The referencegraphics and the guidance control graphics provide guidance for trackingthe physical interventional device through the patient's anatomy usingthe 3D anatomical holographic projections and the 3D holographicrepresentation of the interventional device.

In another aspect, the present disclosure can include a head-mounteddevice to holographic image-guidance for a non-vascular percutaneousprocedure. The head-mounted device includes a non-transitory memory thatis accessed by the processor to execute instructions to performoperations. The operations include receiving tracking data for aphysical interventional device in a tracking coordinate system;transforming the tracking data for the physical interventional device inthe tracking coordinate system into a headset coordinate system;accessing image data from a pre-operative image of a patient's anatomycomprising a physical operative site in an imaging coordinate system;transforming the image data in the imaging coordinate system into theheadset coordinate system; registering a 3D holographic representationof the interventional device based on the tracking data for the physicalinterventional device in the headset coordinate system to 3D anatomicalholographic projections of the patient's anatomy based on the imagingdata in the headset coordinate system; displaying the 3D anatomicalholographic projections providing a visualization of a holographicversion of the patient's anatomy with reference graphics related to aphysical operative site within the patient's anatomy; displaying the 3Dholographic representation of the interventional device providing avisualization of a holographic version of the interventional device withguidance control graphics related to the physical interventional device;and navigating the 3D holographic representation of the interventionaldevice in the 3D anatomical holographic projections based on thetracking data for the interventional device in the headset coordinatesystem. The reference graphics and the guidance control graphics provideguidance for tracking the physical interventional device through thepatient's anatomy using the 3D anatomical holographic projections andthe 3D holographic representation of the interventional device.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features of the present disclosure will becomeapparent to those skilled in the art to which the present disclosurerelates upon reading the following description with reference to theaccompanying drawings, in which:

FIG. 1 is a block diagram illustration showing an example of a systemthat provides holographic image-guidance for non-vascular percutaneousprocedures in accordance with an aspect of the present disclosure;

FIG. 2 is a block diagram illustration showing an example of thecoordinate transformation accomplished by the head-mounted device ofFIG. 1;

FIG. 3 is an illustration of an example of a hologram augmented to ananatomical manikin including a simulated biopsy needle;

FIG. 4 is an illustration of an example of a hologram showing referencegraphics and guidance control graphics used to control navigation of aphysical interventional device through a physical body using holographicimage-guidance; and

FIGS. 5 and 6 are process flow diagrams of example methods for providingholographic image-guidance for non-vascular percutaneous procedures inaccordance with another aspect of the present disclosure.

DETAILED DESCRIPTION I. Definitions

Unless otherwise defined, all technical terms used herein have the samemeaning as commonly understood by one of ordinary skill in the art towhich the present disclosure pertains.

In the context of the present disclosure, the singular forms “a,” “an”and “the” can also include the plural forms, unless the context clearlyindicates otherwise.

As used herein, the terms “comprises” and/or “comprising” can specifythe presence of stated features, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, steps, operations, elements, components, and/or groups.

As used herein, the term “and/or” can include any and all combinationsof one or more of the associated listed items.

Additionally, although the terms “first,” “second,” etc. may be usedherein to describe various elements, these elements should not belimited by these terms. These terms are only used to distinguish oneelement from another. Thus, a “first” element discussed below could alsobe termed a “second” element without departing from the teachings of thepresent disclosure. The sequence of operations (or acts/steps) is notlimited to the order presented in the claims or figures unlessspecifically indicated otherwise.

As used herein, the term “percutaneous” refers to something that ismade, done, or effected through the skin.

As used herein, the term “percutaneous medical procedure” refers toaccessing the internal organs or tissues via needle-puncture of theskin, rather than by using an open approach where the internal organs ortissues are exposed (typically with a scalpel).

As used herein, the term “non-vascular” when used with “percutaneousmedical procedure” refers to a medical procedure performed on anyportion of the subject's body distinct from the vasculature that isaccessed percutaneously. Examples of percutaneous medical procedures caninclude a biopsy, a tissue ablation, a cryotherapy procedure, abrachytherapy procedure, an endovascular procedure, a drainage procedurean orthopedic procedure, a pain management procedure, a vertebroplastyprocedure, a pedicle/screw placement procedure, a guidewire-placementprocedure, a SI-Joint fixation procedure, a training procedure, or thelike.

As used herein, the term “interventional device” refers to a medicalinstrument used during the non-vascular percutaneous medical procedure.

As used herein, the term “tracking system” refers to something used toobserve one or more objects undergoing motion and supply a timelyordered sequence of tracking data (e.g., location data, orientationdata, or the like) in a tracking coordinate system for furtherprocessing. As an example, the tracking system can be an electromagnetictracking system that can observe an interventional device equipped witha sensor-coil as the interventional device moves through a patient'sbody.

As used herein, the term “tracking data” refers to information recordedby the tracking system related to an observation of one or more objectsundergoing motion.

As used herein, the term “tracking coordinate system” refers to a 3DCartesian coordinate system that uses one or more numbers to determinethe position of points or other geometric elements unique to theparticular tracking system. For example, the tracking coordinate systemcan be rotated, scaled, or the like, from a standard 3D Cartesiancoordinate system.

As used herein, the term “head-mounted device” or “headset” refers to adisplay device, configured to be worn on the head, that has one or moredisplay optics (including lenses) in front of one or more eyes. In someinstances, the head-mounted device can also include a non-transitorymemory and a processing unit. An example of a head-mounted device is aMicrosoft HoloLens.

As used herein, the term “headset coordinate system” refers to a 3DCartesian coordinate system that uses one or more numbers to determinethe position of points or other geometric elements unique to theparticular head-mounted device system. For example, the headsetcoordinate system can be rotated, scaled, or the like, from a standard3D Cartesian coordinate system.

As used herein, the term “imaging system” refers to something thatcreates a visual representation of the interior of a patient's body. Forexample, the imaging system can be a computed tomography (CT) system, amagnetic resonance imaging (MRI) system, an ultrasound (US) system, orthe like.

As used herein, the term “image data” refers to information recorded in3D by the imaging system related to an observation of the interior ofthe patient's body. For example, the image data can include tomographicimages represented by data formatted according to the Digital Imagingand Communications in Medicine (DICOM) standard (referred to as DICOMdata herein).

As used herein, the term “imaging coordinate system” refers to a 3DCartesian coordinate system that uses one or more numbers to determinethe position of points or other geometric elements unique to theparticular imaging system. For example, the imaging coordinate systemcan be rotated, scaled, or the like, from a standard 3D Cartesiancoordinate system.

As used herein, the term “hologram”, “holographic projection”, or“holographic representation” refers to a computer-generated imageprojected to a lens of a headset. Generally, a hologram can be generatedsynthetically (in an augmented reality (AR)) and is not related tophysical reality.

As used herein, the term “physical” refers to something real. Somethingthat is physical is not holographic (or not computer-generated).

As used herein, the term “two-dimensional” or “2D” refers to somethingrepresented in two physical dimensions.

As used herein, the term “three-dimensional” or “3D” refers to somethingrepresented in three physical dimensions. An element that is “4D” (e.g.,3D plus a time and/or motion dimension) would be encompassed by thedefinition of three-dimensional or 3D.

As used herein, the term “reference graphics” refers to a holographicimage related to a physical operative site within the patient's anatomyto aid in guidance of an interventional device.

As used herein, the term “guidance control graphics” refers to aholographic image related to an interventional device to aid in guidanceof the interventional device.

As used herein, the term “integrated” can refer to two things beinglinked or coordinated. For example, a coil-sensor can be integrated withan interventional device.

As used herein, the term “degrees-of-freedom” refers to a number ofindependently variable factors. For example, a tracking system can havesix degrees-of-freedom—a 3D point and 3 dimensions of rotation.

As used herein, the term “real-time” refers to the actual time duringwhich a process or event occurs. In other words, a real-time event isdone live (within milliseconds so that results are available immediatelyas feedback). For example, a real-time event can be represented within100 milliseconds of the event occurring.

As used herein, the terms “subject” and “patient” can be usedinterchangeably and refer to any vertebrate organism.

II. Overview

The present disclosure relates generally to non-vascular percutaneousprocedures and, more specifically, to systems and methods that provideholographic image-guidance for the non-vascular percutaneous procedures.The holographic image-guidance allows for real time tracking of aphysical interventional device through a patient's body to aninterventional target. Tracking data (position and orientation) for thephysical interventional device can be captured using a tracking systemto track the physical interventional device through the patient's body.A 3D holographic interventional device can follow the path of thephysical interventional device based on the tracking data and beprojected in 3D within a 3D holographic anatomical image that isgenerated based on pre-operative images. The 3D holographicinterventional device can be displayed within the 3D holographicanatomical image due to the tracking data and the image data are eachtransformed into a coordinate system of a headset that displays the 3Dholographic images.

Such holographic image-guidance is achieved using augmented reality (AR)to display 3D holographic projections at the interventional site inregistration with the physical patient instead of displaying sourceimages on a 2D monitor. The systems and methods of the presentdisclosure help to overcome limitations of traditional image guidedprocedures, which use CT and fluoroscopy. The use of holographicimage-guidance leads to a shorter procedure time with less radiationdose to both the interventionalist and the patient, as well as fewerprocedural complications caused by hitting critical structures duringthe procedure. Moreover, the holographic image-guidance allows forhands-free guidance and navigation that facilitates sterility of thesurgical field. The use of holographic image-guidance can also increaserevenue by allowing percutaneous procedures to be performed outside of ahigh-tech 3D CT suite, using less expensive (lower tech, such as 2D)guidance imaging, such as ultrasound and mobile C-arm fluoroscopy inconjunction with the holographic image-guidance.

III. Systems

One aspect of the present disclosure can include a system 10 (FIG. 1)that provides holographic image-guidance for non-vascular percutaneousprocedures. Non-vascular percutaneous procedures can refer to anymedical procedure performed on any portion of the subject's bodydistinct from the vasculature that is accessed percutaneously. Examplesof non-vascular percutaneous medical procedures can include a biopsy, atissue ablation, a cryotherapy procedure, a brachytherapy procedure, adrainage procedure an orthopedic procedure, a pain management procedure,a vertebroplasty procedure, a pedicle/screw placement procedure, aguidewire-placement procedure, a SI-Joint fixation procedure, a trainingprocedure, or the like.

The holographic image-guidance can use 3D augmented reality to replacetraditional 2D image guidance. The system 10 can include a head-mounteddevice 11 that can be configured to facilitate the 3D augmented realityholographic display. The head-mounted device 11 can include anon-transitory memory 13 and a processing unit 12 (that may include oneor more hardware processors) that can aid in the display of theholographic display. The head-mounted device can also include a camerato record one or more images, one or more image-generation components togenerate/display a visualization of the hologram, and/or othervisualization and/or recording elements.

The head-mounted device 11 can be in communication with a trackingsystem 14 to receive tracking data. The tracking system 14 can be anelectromagnetic (EM) tracking system that can detect the location andorientation of a physical interventional device. The physicalinterventional device can be integrated with one or more sensor-coils,and the tracking system 14 can determine the location and orientation ofthe one or more sensor-coils, which can correlate to the location andorientation of the physical interventional device. For a non-rigiddevice, one or more sensor-coils can be located at a tip of the physicalinterventional device. However, for a rigid device the one or moresensor-coils can be located anywhere along the physical interventionaldevice and need not be on the physical interventional device at all(e.g., may be located outside the patient's body). As the physicalinterventional device traverses a patient's body, the tracking system 14can detect the one or more sensor-coils and provide tracking data (e.g.,with six degrees of freedom) in response to the detection. For example,the tracking data can include real-time 3D position data and real-time3D orientation data. The tracking system can also detect coil-sensorsthat are not located on the physical interventional device (e.g.,located on fiducial markers or other imaging targets). The tracking datacan be recorded in a coordinate system of the tracking system 14 andsent (wirelessly and/or via a wired connection) to the head-mounteddevice 11.

The head-mounted device 11 can also be in communication with a computingdevice 15 to receive data related to a preoperative imaging study of atleast a portion of the underlying anatomy. The preoperative imagingstudy can record 3D images (e.g., tomographic images) of the portion ofthe patient's anatomy. The 3D images can be represented by imaging data(which can be DICOM data), which can be formatted according to animaging coordinate system of the certain imaging modality that was usedto record the imaging data and sent to the head-mounted device 11.

As shown in FIG. 2, the tracking coordinate system 16 and the imagingcoordinate system 18 can each be transformed (e.g., translated androtated) into the headset coordinate system 17. The transformation canbe based on a rigid body affine transformation, for example.Accordingly, the tracking data in the tracking coordinate system 16 canbe transformed (e.g., translated and rotated) into the headsetcoordinate system 18. Likewise, the imaging data in the imagingcoordinate system 18 can be transformed (e.g., translated and rotated)into the headset coordinate system 18. Only when the tracking data andthe imaging data is each transformed into the headset coordinate systemcan a visualization be generated showing a 3D holographic viewillustrating the navigation of the physical interventional device withinthe patient's body.

FIGS. 3 and 4 each are images showing visualizations of the holographicimage-guidance. FIG. 3 shows a hologram augmented to an anatomicalmanikin including a simulated biopsy needle 32. FIG. 4 shows a hologramof a trocar being placed within a target tissue. The hologram includesguidance graphics and annotations showing tracking information. However,the guidance can be visual and/or auditory feedback related to locationand orientation of the physical interventional device. FIG. 4 also showsthree non-collinear fiducial markers 42 that are used for registration.The fiducial markers 42 can be combination markers so that the positionsof the fiducial markers 42 can be detected in the tracking coordinatesystem and the headset coordinate system. Additionally, positions of thefiducial markers 42 can be located (e.g., sensor coils on the physicalpatient's skin). The positions of the fiducial markers can be matched tothree or more locations in the image coordinate system.

The head-mounted device 11 can further transform the visualization ofthe holographic image-guidance. For example, the transformation can betranslated, rotated, and/or scaled. The transformed visualization,however, would no longer be precisely aligned with the patient's body.

IV. Methods

Another aspect of the present disclosure can include methods 50, 60(FIGS. 5 and 6) for providing holographic image-guidance for thenon-vascular percutaneous procedures. The methods 50, 60 can be executedby hardware—for example, by the head-mounted device 11 shown in FIG. 1and described above.

The methods 50 and 60 are illustrated as process flow diagrams withflowchart illustrations. For purposes of simplicity, the methods 50 and60 shown and described as being executed serially; however, it is to beunderstood and appreciated that the present disclosure is not limited bythe illustrated order as some steps could occur in different ordersand/or concurrently with other steps shown and described herein.Moreover, not all illustrated aspects may be required to implement themethods 50 and 60. Additionally, one or more elements that implement themethods 50 and 60, such as head-mounted device 11 of FIG. 1, may includea non-transitory memory 13 and one or more processors (processing unit12) that can facilitate the holographic image-guidance.

Referring now to FIG. 5, illustrated is a method 50 for providing andusing the holographic image-guidance. The holographic image-guidance canbe used to guide an interventional device to a target within a patient'sbody without requiring intraoperative imaging and/or used adjunct tointraoperative imaging. The steps of the method 50 can be performed bythe head-mounted device 11 of FIG. 1.

At step 52, 3D anatomical holographic projections of a patient's anatomycan be displayed. The holographic projections can include referencegraphics related to a physical operative site with a patient's anatomy.At step 54, a holographic representation of a physical interventionaldevice can be displayed. The holographic representation can includeguidance control graphics related to the physical interventional device.At step 56, the 3D holographic representation of the interventionaldevice can be navigated through the 3D anatomical holographicprojection. The reference graphics and the guidance control graphics canprovide guidance (e.g., visual guidance (pictorial, type, annotation,etc.) and/or auditory guidance) for tracking the physical interventionaldevice through the patient's anatomy using the holographic guidance(using the 3D anatomical holographic projections and the 3D holographicrepresentation of the interventional device). For example, when a lineassociated with the reference graphics and a line associated with theguidance control graphics intersect, the physical interventional devicecan be in alignment with a trajectory that would intersect an anatomicaltarget tissue. This can be accompanied by a holographic annotation thatreports the distance and/or angle deviation from a targeted position ororientation.

The reference graphics and the guidance control graphics can be used toprovide event driven guidance. For example, when a trocar is within thepatient's body, the reference graphics and the guidance control graphicscan provide auditory and/or visual guidance as the trocar moves. As thetrocar is moved through the patient's body, a beep can be used toindicate proximity to a target. Similarly, graphics can providereal-time annotations of the position and the angle of the trocar and/orshowing the intersection with the target.

The 3D anatomical holographic projections and the 3D holographicrepresentation of the interventional device are expressed in a commoncoordinate system (the 3D headset coordinate system). The 3D anatomicalholographic projections are created based on image data that isoriginally in an imaging coordinate system. Similarly, the 3Dholographic representation of the interventional device is trackedthrough the 3D anatomical holographic projections based on tracking datathat is originally in a tracking coordinate system. FIG. 6 shows amethod 60 for registering the coordinates from a tracking system and animaging system to the common coordinate system (the coordinate systemused by a head-mounted device to generate the holograms). Also includedare three non-collinear fiducial markers 42 (e.g., sensor-coils that canbe detected by the tracking system and the headset) that are used forregistration.

A physical interventional device can be integrated with one or moresensor-coils. For a non-rigid device, one or more sensor-coils can belocated at a tip of the physical interventional device. However, for arigid device the sensor-coils are can be located at any position alongthe physical interventional device (or even outside the patient's body).As the physical interventional device traverses a patient's body, thetracking system (e.g., an electromagnetic tracking system) can samplethe one or more sensor-coils and provide tracking data (e.g., with sixdegrees of freedom) in response to the detection. For example, thetracking data can include real-time 3D position data and real-time 3Dorientation data. The tracking data, in the tracking coordinate system,can be transmitted to the head-mounted device. At step 62, tracking datafor a physical interventional device in tracking coordinates can betransformed into a headset coordinate system (by the head-mounteddevice).

A patient can undergo a preoperative imaging study that images at leasta portion of the underlying anatomy. The preoperative imaging studiescan record 3D images (e.g., tomographic images) of the portion of thepatient's anatomy. The 3D images can be represented by imaging data(which can be DICOM data), which can be formatted according to animaging coordinate system of the certain imaging modality that was usedto record the imaging data and sent to the head-mounted device. At step64, image data in imaging coordinates can be transformed to the headsetcoordinate system (by the head-mounted device). 3D anatomicalhierarchical projections generated based on the image data can be basedon one or more surface mesh models, multi-planar reformatted images, orthe like.

At step 66, a visualization can be rendered (by the head-mounted device)using the tracking data in the headset coordinates and the imaging datain the headset coordinates. The hologram can include a 3D anatomicalholographic projection based on the imaging data transformed to theheadset coordinates and a 3D holographic representation of theinterventional device based on the tracking coordinates. As previouslynoted, graphics, including the reference graphics and the guidancecontrol graphics, can provide guidance for tracking the physicalinterventional device through the patient's anatomy using theholographic guidance (using the 3D anatomical holographic projectionsand the 3D holographic representation of the interventional device). Thevisualization can be transformed by translating, rotating, and/orscaling to enhance the navigation. The transforming can be triggered bya physical movement of the head-mounted device (e.g., by tilting thehead in a particular manner).

V. Example Registration Technique

The following description describes an example registration techniquethat can be used to register tracking data (in a tracking coordinatesystem) and imaging data (in an imaging coordinate system) into a commonholographic coordinate system utilized by the head-mounted device thatprovides the holograms. It will be noted that this is a description ofjust a single example and other registration techniques can be usedwithin the scope of this disclosure.

The registration technique transforms the tracking data and the imagingdata into the headset coordinate system. To do so, the head-mounteddevice can perform two affine transformations: EM-to-HL (or trackingsystem to head-mounted device) and U-to-HL (or imaging system tohead-mounted device). In this example, the transformations rely onco-location of image targets (e.g., fiducial markers placed on thepatient's skin and/or anatomical targets) in the different coordinatesystems. Three or more image targets can be used that are non-collinear.For example, at least some the image targets must be visible to thetracking system (requiring a sensor-coil on each of the image targets)and a camera associated with the head-mounted device. In some instances,the same or others of the image targets must be visible to thepre-operative imaging system and the camera associated with the headmounted device.

For the EM to HL transformation, two 3D point sets P_(EM) andP_(H, image target) consisting of corresponding (co-located) point pairsare used: (1) P_(i)(x_(hl), y_(hl), z_(hl)) in HL coordinates locatedwith image targets via the head-mounted device and (2) P_(i)(x_(EM),y_(EM), z_(EM)) localized with EM sensors (physically associated withthe image targets) in EM coordinates for 4 corresponding points. The 3Dpoint sets (P_(EM), P_(H, image target)) are used to determine the EM toHL transformation ([^(EM)M_(HL)]=LSM{P_(EM), P_(H, image target)}) on aprocessing unit associated with the head-mounted device using aleast-square method (LSM). Position and orientation data from the imagetargets (referred to as landmark sensors) and sensor-coils on theinterventional device (P_(EM,lm,i)(t) [^(EM)M_(HL,cal)]=P_(HL,lm,i)(t))are then transformed into HL coordinates using [^(EM)M_(HL)]. Pleasenote that the LSM method is not the exclusive method that can be used.

With the locations of the landmark sensors and the sensor-coils on theinterventional device tracked in HL coordinates the corresponding 3Dpoint sets in the imaging (U) coordinates for the markers and devicesensors can also be transformed to HL coordinates. For this, the LSMwill also be used to determine a transformation [^(U)M_(HL)(t)], betweenimaging coordinates and HL coordinates ([^(U)M_(HL)]=LSM{P_(U, lm&d,i,),P_(HL,lm,i)}). [^(U)M_(HL)(t)] is then used to transform the imagingcoordinates to HL coordinates. This allows the head-mounted device toproject the holograms on the patient's body (e.g., in the surgicalfield).

From the above description, those skilled in the art will perceiveimprovements, changes and modifications. Such improvements, changes andmodifications are within the skill of one in the art and are intended tobe covered by the appended claims. All patents, patent applications, andpublications cited herein are incorporated by reference in theirentirety.

The following is claimed:
 1. A method comprising: receiving, by a head-mounted device comprising a processor, tracking data for a physical interventional device in a tracking coordinate system, wherein the physical interventional device is used during a percutaneous non-vascular medical procedure; transforming, by the head-mounted device, the tracking data for the physical interventional device in the tracking coordinate system into a headset coordinate system; accessing, by the head-mounted device, image data from a pre-operative image of a patient's anatomy comprising a physical operative site in an imaging coordinate system; transforming, by the head mounted device, the image data in the imaging coordinate system into the headset coordinate system; registering, by the head-mounted device, a 3D holographic representation of the interventional device based on the tracking data for the physical interventional device in the headset coordinate system to 3D anatomical holographic projections of the patient's anatomy based on the imaging data in the headset coordinate system; displaying, by the head mounted device, the 3D anatomical holographic projections providing a visualization of a holographic version of the patient's anatomy including a physical operative site within the patient's anatomy; and navigating, by the head mounted device, the 3D holographic representation of the interventional device in the 3D anatomical holographic projections based on the tracking data for the interventional device in the headset coordinate system.
 2. The method of claim 1, wherein the physical interventional device comprises an integrated sensor-coil that is detectable by a tracking system to provide the tracking data in the tracking coordinate system.
 3. The method of claim 2, wherein the tracking data has six-degrees of freedom (6 DOF).
 4. The method of claim 2, wherein the tracking data comprises real-time 3D position data and real-time 3D orientation data.
 5. The method of claim 2, wherein the sensor coil is located at a tip of the physical interventional device.
 6. The method of claim 1, wherein the image data from the pre-operative image of a patient's anatomy is represented by patient-specific DICOM image data from one or more 3D pre-operative tomographic image data sets.
 7. The method of claim 1, wherein the 3D anatomical holographic projections are based on one or more surface mesh models or multi-planar reformatted images created from the image data.
 8. The method of claim 1, further comprising transforming, by the head mounted device, a visualization of the 3D anatomical holographic projections by at least one of translating, rotating, and scaling to enhance the navigating.
 9. The method of claim 8, wherein the transforming is triggered by a physical movement of the head mounted device.
 10. The method of claim 1, wherein the transforming further comprises: locating positions of three or more fiducial markers on the physical patient's anatomy, wherein the positions are in the tracking coordinate system; and matching the positions of the three or more fiducial markers to three or more locations in the image data in the image coordinate system.
 11. The method of claim 10, wherein the positions of the three or more fiducial markers are non-collinear.
 12. The method of claim 10, wherein the three or more fiducial markers are sensor-coils placed on the patient's skin.
 13. The method of claim 1, wherein the percutaneous non-vascular medical procedure comprises a biopsy, a tissue ablation, a cryotherapy procedure, a brachytherapy procedure, a drainage procedure, an orthopedic procedure, a pain management procedure, a pedicle/screw placement procedure, a guidewire placement procedure, a SI-Joint fixation procedure, or a training procedure.
 14. The method of claim 1, wherein the guidance comprises visual feedback or auditory feedback related to location and orientation of the physical interventional device.
 15. The method of claim 14, further comprising providing the visual feedback to monitor when a graphic corresponding to the guidance control graphics intersects a graphic corresponding to the reference graphics indicating that the physical interventional device is in alignment with a trajectory that would place the physical interventional device at a target within the patient's body.
 16. The method of claim 15, wherein the visual feedback includes a holographic annotation that reports the distance or angle deviation from a targeted position or orientation.
 17. A head-mounted device comprising: a non-transitory memory storing instructions; and a processor to access the non-transitory memory and execute the instructions to: receive tracking data for a physical interventional device in a tracking coordinate system, wherein the physical interventional device is used during a percutaneous non-vascular medical procedure; transform the tracking data for the physical interventional device in the tracking coordinate system into a headset coordinate system; access image data from a pre-operative image of a patient's anatomy comprising a physical operative site in an imaging coordinate system; transform the image data in the imaging coordinate system into the headset coordinate system; register a 3D holographic representation of the interventional device based on the tracking data for the physical interventional device in the headset coordinate system to 3D anatomical holographic projections of the patient's anatomy based on the imaging data in the headset coordinate system; displaying, by the head mounted device, the 3D holographic representation of the interventional device providing a visualization of a holographic version of the interventional device; and navigating the 3D holographic representation of the interventional device in the 3D anatomical holographic projections based on the tracking data for the interventional device in the headset coordinate system.
 18. The head-mounted device of claim 17, further comprising a head mounted display to display the visualization.
 19. The head-mounted device of claim 18, wherein the processor further executes the instructions to transform the visualization by at least one of translating, rotating, and scaling to enhance the navigating triggered by a physical movement of the head mounted device.
 20. The head-mounted device of claim 17, wherein the transforming further comprises: locating positions of three or more fiducial markers on the physical patient's skin, wherein the positions are in the tracking coordinate system, wherein the positions are non-collinear; and matching the positions of the three or more fiducial markers to three or more locations in the image data in the image coordinate system. 